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Background 


The  concept  of  reliability  is  central  to  any  evaluation  of  an  assessment 
instrument.  The  importance  of  reliability  can  be  seen  in  two  ways.  First, 
by  definition,  reliability  means  consistency,  repeatability,  and  the  sense 
of  confidence  we  can  have  in  any  survey  scores.  As  Miller  (1966)  notes, 
"Reliability  is  the  extent  to  which  measuring  operations  correlate  with 

themselves  when  applied  repeatedly  to  the  same  subjects”  (p.  17).  Lyman 

(1971)  says  reliability  refers  to  the  "reproducibility"  of  results  (p.  25). 
Finally,  Cronbach  (1960)  believes  reliability  "always  refers  to  consistency 
throughout  a  series  of  measurements"  (p.  126).  Reliability  is  an  important 
measure,  then,  of  how  much  trust  can  be  put  in  results  of  a  survey  or 

instrument.  A  survey  that  gives  different  results  every  time  even  when 

administered  to  the  same  people  under  the  same  conditions  would  not  be 
usable  and  results  would  not  be  Interpretable. 

Second,  reliability  and  validity  are  closely  related.  Stanley  and 
Hopkins  (1972)  point  out,  "Reliability  is  a  necessary  but  not  sufficient 
condition  for  validity"  (p.  114).  Cronbach  (1960)  further  defines  this  re¬ 
lationship  by  stating  the  "validity  coefficient  cannot  exceed  the  square 
root  of  the  reliability  coefficient"  (p.129).  Validity,  then,  is  simply  the 
ability  of  the  Instrument  to  do  what  we  want  it  to  do  (Lyman,  1971);  it  is 
the  most  essential  feature  of  an  instrument.  Since  reliability  limits 
validity  and  Is  a  prerequisite  for  validity,  accurate  measurement  of  relia¬ 
bility  is  an  Important  consideration. 

Methods  of  Assessing  Reliability 

Several  methods  of  assessing  reliability  are  available.  Carmines  and 
Zeller  (1979)  list  four  such  methods:  the  retest  method  (stability);  the 


alternative- form  method;  the  split-halves  method;  and  the  internal  consis¬ 
tency  methods,  primarily  Kuder-Richardson  20  and  21  (KR20  and  KR21)  and 
Cronbach's  alpha.  Of  these  four  methods,  two  are  appropriate  in  the  present 
situation  and  two  are  not.  The  alternative  form  method  is  not  applicable 
here  since  the  instrument  in  question,  the  Organizational  Assessment  Package 
( OAP ) ,  exists  in  only  one  form.  Similarly,  the  split-half  method  is  not 
applicable  because  of  accuracy  problems.  Split-half  coefficients  are  calcu¬ 
lated  by  dividing  items  in  half  and  correlating  one  half  of  the  items  with 
the  other  half  of  the  items.  This  reduction  in  number  of  items  correlated 
spuriously  lowers  reliability  coefficients  (Cronbach,  1960).  The  split-half 
method,  therefore,  must  use  a  formula  such  as  the  Spearman-Brown  correction 
formula  to  estimate  the  full-scale  reliability  from  the  half-scale  coeffi¬ 
cient.  The  use  of  this  formula  adds  more  inconsistency,  since  it  is  only  an 
estimate  of  the  original  full-scale  reliability.  This  is  an  especially  ser¬ 
ious  problem  for  an  instrument  such  as  the  OAP  that  provides  factor  rather 
than  full  scale  scores. 

Retest  reliability  (stability),  however,  is  necessary  to  calculate 
because  of  the  way  the  OAP  is  used.  The  OAP  is  used  as  both  a  data  gather¬ 
ing  and  consulting  evaluation  tool.  As  a  result,  the  stability  of  OAP 
results  for  approximately  six  week  and  six  month  Intervals  is  important 
information.  Used  alone  however,  retest  coefficients  are  insufficient,  and 
high  coefficients  alone  should  not  be  taken  as  an  indication  of  good  reli¬ 
ability  (Nunally  and  Durham,  1975).  Therefore,  use  of  a  second  reliability 
assessment  method,  internal  consistency,  provides  important  additional  veri¬ 
fication.  Among  the  available  options,  the  KR20  and  KR21  procedures  are 
inappropriate  since  the  OAP  items  are  not  scored  according  to  an  all  or  none 
(right  or  wrong)  system  (Anastasi,  1976).  Cronbach's  alpha,  however,  pro¬ 
vides  an  internal  consistency  measure  for  multiple  scored  items.  The 


retest  and  Cronbach's  alpha  procedures  appear,  therefore,  to  be  the  most 
appropriate  methods  to  assess  OAP  reliability.  Further  comments  on  both  of 
these  methods  follow. 

Retest  Method.  As  briefly  mentioned,  retest  reliability  is  important  to 
calculate  because  the  OAP  is  used  both  as  a  diagnostic  tool  and  as  an  evalua¬ 
tion  tool.  This  information  is  necessary,  for  example,  to  assist  in  develop¬ 
ing  equations  to  help  control  for  regression  toward  the  mean,  a  threat  to  the 
internal  validity  of  consulting  evaluation  results  (Campbell  and  Stanley, 
1963).  Determining  the  stability  of  scores  over  approximately  a  six  week 
interval  (roughly  the  period  between  data  collection  and  feedback)  and  a  six 
month  interval  (roughly  the  period  between  feedback  and  evaluation)  was  an 
appropriate  starting  place  to  begin  gathering  this  information. 

Although  the  retest  method  seems  a  reasonable  approach  to  determining 
validity,  it  does  have  some  drawbacks  as  discussed  by  Carmines  and  Zeller 
(1979).  First,  evaluation  requires  two  measurements  of  the  same  individuals. 
This  requirement  may  lead  to  increased  expense  and  other  practical  problems. 
Further,  the  optimum  time  interval  between  testings  is  often  difficult  to 
determine.  If  the  interval  is  too  short,  coefficients  may  be  inflated  due  to 
memory  effects  after  the  pretest.  If  the  interval  is  too  long,  the  environ¬ 
mental  situation  or  even  the  underlying  theoretical  construct  may  change, 
resulting  in  underestimates  of  reliability.  Finally,  the  problem  of  reactiv¬ 
ity  or  pretest  sensitization  (Bracht  and  Glass,  1968)  may  occur.  In  this 
case,  the  change  from  pretest  to  posttest  can  be  caused  just  by  the  pretest. 
This  seems  particularly  likely  in  an  organizational  setting  when  data  collec¬ 
tion  causes  and  directs  energy  (Nadler,  1977)  and  especially  costly  in  a  set¬ 
ting  where  a  primary  goal  is  collection  of  valid  data  (Argyris,  1970).  These 
problems  do  not  eliminate  the  necessity  for  stability  coefficients,  but  they 
do  point  out  the  need  for  an  additional  method  of  determining  factor  relia¬ 
bility. 
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Internal  Consistency  Method:  Cronbach's  Alpha.  Generally  the  most  popu¬ 
lar  of  the  internal  consistency  methods,  Cronbach's  alpha  can  be  obtained 
from  a  single  survey  administration  and  eliminates  the  inconsistency  of 
splitting  items.  Its  calculation  is  based  on  the  number  of  items  in  a  scale 
or  factor  and  the  mean  interitem  correlation  for  that  same  scale  or  factor. 
Usually,  therefore,  as  the  average  interitem  correlation  and/or  the  number  of 
items  increase,  so  does  the  value  of  alpha.  These  procedures  must  be  bal¬ 
anced,  however.  For  example,  there  is  an  upper  bound  on  significant  in¬ 
creases  in  alpha  from  adding  items.  In  addition,  adding  items  that  reduce 
the  interitem  correlation  will  not  increase  alpha. 

It  should  also  be  noted  that  alpha  is  often  considered  the  lower  bound 
of  internal  consistency  reliability.  Thus,  alpha  may  generally  be  consid¬ 
ered  a  conservative  estimate  of  the  true  reliability  of  a  scale  or  factor 
««» 

(Carmines  and  Zeller,  1979). 

Purpose  of  the  Report 

With  the  previous  background  in  mind,  the  present  report  has  two  pur¬ 
poses.  The  first  purpose  is  assessing  the  current  reliability  of  OAP  fac¬ 
tors.  Cronbach's  alpha  coefficients  have  not  been  formally  studied  since  the 
original  OAP  development  (Hendrix  and  Halverson,  1979),  and  retest  coeffi¬ 
cients  have  never  been  studied  and  published.  For  both  reasons,  new  informa¬ 
tion  was  needed.  The  second  purpose  of  the  report  is  anchored  to  the  future 
of  the  OAP.  The  instrument  is  currently  undergoing  a  complete  factor-by¬ 
factor  revision  after  two  years  of  use.  The  current  report  will  provide 
important  information  helpful  for  decisions  about  which  items  and  factors  to 
retain,  revise,  or  delete. 

In  genera)  format,  the  report  consists  of  two  studies,  each  using  a  dif¬ 
ferent  sample  derived  in  different  ways.  As  such,  the  methodology  and 
results  of  each  will  be  reported  separately. 
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Method 


Study  1 

Sample.  The  sample  for  this  study  consisted  of  19  Air  Force  personnel 
attending  the  Academic  Instructor  School  (AIS)  at  Maxwell  Air  Force  Base, 
Alabama.  Of  the  19  people,  18  were  male,  one  was  female;  16  were  enlisted, 
three  were  officers.  Among  the  officers  were  one  each  second  lieutenant, 
captain  and  lieutenant  colonel.  Among  the  enlisted  people,  five  were  staff 
sergeants,  five  were  technical  sergeants,  five  were  master  sergeants,  and 
one  was  a  senior  master  sergeant.  The  average  age  of  the  people  in  the 
sample  was  33  years. 

The  total  group  of  19  represented  approximately  48%  attrition  from  a 
beginning  sample  of  36  personnel  who  took  the  survey  the  first  time.  All  36 
were  volunteers  who  came  after  normal  class  periods  to  take  the  survey.  Of 
the  original  36,  the  current  19  returned  to  take  the  survey  a  second  time. 
In  some  cases,  the  number  of  people  (N)  may  go  lower  due  to  missing  re¬ 
sponses. 

Instrumentation.  The  OAP  is  a  109  question  survey  designed  by  the  Air 
Force  Human  Resources  Laboratory  to  aid  the  Leadership  Management  and  Devel¬ 
opment  Center  (LMDC)  in  its  mission  to:  (a)  provide  management  consulting 
services  to  Air  Force  commanders  upon  request,  (b)  to  provide  leadership  and 
management  training,  and  (c)  to  conduct  research  on  Air  Force  systemic 
Issues  with  information  within  the  accumulated  data  base  (Hendrix  and 
Halverson,  1979). 

Administration  of  the  survey  is  the  first  step  in  the  consultation  pro¬ 
cess.  The  survey  is  given  to  a  stratified  random  sample  of  the  organization 
to  which  LMDC  has  been  invited.  The  results  of  the  survey  are  an  important 
feature  in  the  assessment  of  the  organization.  The  results  are  handled  in  a 


confidential  manner  between  LMOC  and  the  client.  After  approximately  five 
to  six  weeks  for  analysis,  feedback  of  data  is  then  provided  to  commanders 
and  supervisors  within  the  organization. 

When  specific  problems  are  encountered,  a  consultant  and  supervisor 
develop  a  management  action  plan  designed  to  reduce  the  problem  at  that 
level  of  the  organization.  Within  six  months,  the  consulting  team  returns 
to  readminister  the  survey  instrument  as  a  means  to  help  assess  the  impact 
of  the  consulting  process. 

The  data  from  each  consulting  effort  are  stored  in  a  cumulating  data 
base  for  research  purposes.  These  data  are  aggregated  by  work  group  codes 
developed  for  this  instrument.  The  data  may  be  recalled  by  demographics 
such  as  personnel  category,  age,  sex,  Air  Force  Specialty  Code  (AFSC),  pay 
grade,  time  in  service,  and  educational  level.  Through  factor  analysis, 
the  93  attitudinal  items  are  combined  into  24  measures  which  cover  job  con¬ 
tent  factors,  job  interferences,  and  various  types  of  supervisory  and 
organizational  climates. 

Procedure.  The  volunteer  AIS  students  were  requested  to  take  the  OAP  at 
the  beginning  and  at  the  end  of  their  school ,  an  interval  of  approximately 
five  weeks.  The  OAP  (Appendix  A)  was  administered  using  the  standard 
response  sheet  (Appendix  B)  but  with  modifications  of  the  standard  adminis¬ 
tration  script  (standard  script  appears  as  Appendix  C).  The  volunteers  were 
told  the  purpose  of  the  survey  was  to  help  LMDC  personnel  learn  more  about 
the  OAP,  and  that  their  help  was  much  appreciated.  Concerning  the  demo¬ 
graphic  factors,  four  were  completed  and  coded  as  usual:  sex,  age,  pay 
grade  and  duty  AFSC.  For  duty  AFSC,  all  persons  were  asked  to  enter  the 
AFSC  of  the  job  they  left  when  they  came  to  AIS.  In  addition,  this  was  the 
job  situation  each  person  was  asked  to  recall  on  both  test  and  retest  when 
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responding  to  the  OAP  content  items.  The  blocks  for  supervisor's  code  were 
left  blank,  work  group  code  was  coded  as  all  ones,  and  each  person's  seminar 
number  was  coded  in  the  first  three  columns  of  the  primary  AFSC  block.  The 
remaining  four  columns  were  left  blank.  For  each  entry,  each  person  dark¬ 
ened  the  corresponding  oval  beneath  that  entry.  This  information  was  used 
from  test  to  retest  for  purposes  of  matching  a  person  by  administration  time 
without  identifying  an  individual.  The  graphic  showing  changes  in  instruc¬ 
tions  is  contained  at  Appendidix  D. 

After  completion  of  the  demographics  on  the  answer  sheet,  standard  in¬ 
structions  were  given  for  completion  of  the  OAP  attitudinal  items.  Everyone 
was  given  as  much  time  as  was  needed  to  complete  the  survey  on  both  occa¬ 
sions,  and  completed  survey  response  sheets  were  deposited  in  a  central 
location.  Both  administrations  of  the  OAP  took  place  in  the  AIS  auditorium 
under  ideal  environmental  conditions. 

Data  were  analy2ed  two  ways.  For  test-retest  analysis,  a  Pearson  pro¬ 
duct  moment  correlation  coefficient  was  calculated  between  results  of  the 
two  administrations.  For  internal  consistency,  a  Cronbach's  alpha  procedure 
was  computed  on  both  test  and  retest  samples.  Both  statistics  were  computed 
using  the  Statistical  Package  for  the  Social  Sciences  (Nie,  et  al ,  1975; 
Hull  &  Nie,  1979). 

For  the  test-retest  correlation,  a  standard  level  of  significance  was 
used  (p  <  .05).  For  the  Cronbach's  alpha,  however,  a  more  direct  standard 
of  comparison  was  used.  Hendrix  and  Halverson  (1979)  noted  that  a  coeffi¬ 
cient  of  .70  or  above  indicates  "...Factors  which  are  reliable..."  ( p .22 ) . 
Carmines  and  Zeller  (1979)  hold  that  "...reliabilities  should  not  be  below 
.80  for  widely  used  scales."  { p .51 ) .  For  purposes  of  this  study,  alpha 
coefficients  were  considered  unacceptable  if  they  were  below  .70  on  either 
test  or  retest.  This  was  also  taken  as  evidence  of  need  for  factor  revi¬ 
sion.  Alpha  coefficients  were  considered  acceptable,  however,  if  they 
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exceeded  .70  on  both  test  and  retest,  good  if  they  exceeded  .80  on  both  test 
and  retest  and  high  if  both  coefficients  exceeded  .90.  In  addition,  any 
factor  with  alpha  coefficients  varying  by  more  than  plus  or  minus  .10  from 
test  to  retest  was  considered  as  problematic. 

For  purpose  of  this  study,  only  OAP  primary  factors  were  used.  Other 
factors  used  by  consultants  are  simply  additive  or  multiplicative  combina¬ 
tions  of  items  represented  by  other  factors,  or,  in  some  cases,  are  combina¬ 
tions  of  other  factors.  Completing  calculations  for  these  "factors"  was 
considered  repetitious  and  possibly  misleading. 

Study  2 

Sample.  The  sample  for  this  study  consisted  of  399  Air  Force  personnel 
assigned  to  Air  Force  Security  Police  squadrons  at  six  installations  within 
the  continental  United  States.  Average  age  of  the  people  in  the  sample  was 
23  years.  Of  the  399  people,  392  were  male  and  7  female.  Racially,  83.0% 
were  White,  not  Hispanic;  11.0%  were  Black,  not  Hispanic;  2.5%  were 
Hispanic;  2.0%  were  Other;  .8%  were  Native  American;  and  .8%  were  Asian- 
Pacific  Islanders.  A  total  of  39.9%  of  the  respondents  had  less  than  two 
years  military  service  and  24.5%  had  four  years  or  more  service  in  the  Air 
Force.  Of  the  399  total,  98.7%  (390)  were  enlisted,  1.0%  (4)  were  officers, 
and  .3%  (1)  was  a  civilian. 

Instrumentation.  The  instrument  used  here  is  still  the  OAP.  The  infor¬ 
mation  is  identical  to  that  contained  in  the  instrumentation  section  of 
Study  1. 

Procedure.  The  original  population  for  this  study  consisted  of  approxi¬ 
mately  3200  Air  Force  personnel  assigned  to  Air  Force  Security  Police  squad¬ 
rons  at  six  installations  within  the  continental  United  States.  This  group 
included  personnel  assigned  to  units  which  were  considered  as  a  control 
group  and  also  personnel  assigned  to  units  which  were  a  test  group  undergo¬ 
ing  an  organizational  redesign.  The  personnel  for  Study  2  came  only  from 
the  control  group,  which  consisted  of  approximately  1600  people. 
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Since  individual  OAP  respondents  are  not  identified  during  administra¬ 
tion,  it  was  necessary  to  derive  some  methodology  for  matching  responses 
gathered  prior  to  and  after  the  redesign.  The  method  chosen  was  to  take  a 
rather  conservative  approach  on  matching  using  selected  demographics  con¬ 
tained  in  the  first  sixteen  questions  of  the  OAP.  The  items  chosen  for 
exact  match  included  base,  personnel  category  (officer,  enlisted,  civilian), 
sex,  marital  status,  ethnic  group,  education  level,  the  first  three  digits 
of  the  alpha  numeric  code  indicating  work  group  of  assignment  within  the 
organization,  and  the  first  three  digits  of  the  numeric  code  identifying  the 
Air  Force  specialty  code  of  the  respondent.  In  addition,  the  data  were  also 
matched  on  demographics  which  may  have  varied  during  the  six  month  period 
from  one  OAP  administration  to  the  next.  Thus,  the  demographics  of  age, 
total  years  in  the  Air  Force,  total  months  in  present  career  field,  total 
months  at  the  current  duty  station,  and  military  pay  grade  were  allowed  to 
match  either  exactly  or  vary  by  one  response  alternative  to  allow  for  promo¬ 
tion  or  the  passage  of  time.  Reduction  of  the  sample  size  from  1600  to  399 
may  seem  extreme.  However,  it  was  better  to  be  conservative;  whenever  more 
than  one  "post"  match  for  a  "pre"  survey  was  encountered,  all  of  the  suspect 
records  were  eliminated.  This  undoubtedly  resulted  in  substantially  fewer 
matches  than  would  otherwise  be  expected. 

The  same  method  of  data  analysis  and  criteria  for  results  were  used  in 
Study  2  as  were  used  Study  1  (see  Procedure  section.  Study  1)  with  one 
exception.  Because  of  the  Increased  sample  size,  a  significance  level  of 
less  than  .001  (p  <  .001)  seemed  appropriate  to  help  Insure  practically  sig¬ 
nificant  as  well  as  statistically  significant  results.  Again,  as  in  Study 
1,  only  primary  OAP  factors  were  used. 


Results 


Study  1 

The  results  of  the  first  study  may  be  seen  in  Tables  1  and  2.  From 
Table  1,  all  test-retest  coefficients  were  significant  at  p  <  .05  level  or 
better.  Of  these,  however,  three  factors  were  lower  than  the  others  and 
seem  to  warrant  further  research.  These  three  were  Factor  804  (Job  Feed¬ 
back),  810  (Job  Performance  Goals),  and  821  (Work  Group  effectiveness).  Two 
of  these  three  factors  also  showed  significant  differences  between  test  and 
retest  means.  In  addition  to  the  correlation  coefficients.  Table  1  shows 
the  means,  standard  deviations,  and  standard  errors  of  measurement  for  both 
test  and  retest,  the  sample  size  for  each  factor  and  results  of  correlated 
means  t-tests  (DuBois,  1965).  Despite  the  convention  of  a  capital  N  being 
reserved  for  discussion  of  population  parameters  and  a  lower  case  n  dealing 
with  sample  statistics,  the  capital  N  is  used  throughout  the  report  to  de¬ 
note  sample  size.  The  lower  case  n  will  denote  number  of  items  in  a  factor 
in  the  reporting  of  Cronbach's  alpha  results. 

Table  2  shows  the  results  of  the  Cronbach's  alpha  analysis  on  the  same 
data.  The  sample  size,  coefficient  alpha,  average  interitem  correlation, 
and  number  of  items  per  factor  are  reported  for  both  test  and  retest  condi¬ 
tions.  Based  on  criteria  set  forth  in  the  methodology  section,  five  factors 
showed  a  coefficient  below  .70  on  either  test  or  retest  and  need  revision 
work.  These  are  Factor  800  (Skill  Variety),  Factor  801  (Task  Identity), 
Factor  804  (Job  Feedback),  Factor  805  (Work  Support)  and  Factor  810  (Job 
Performance  Goals).  An  additional  factor,  816  (Desired  Repetitive  Easy 
Tasks),  did  not  show  either  coefficient  to  be  above  .70.  Under  the 
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Table  1 


Pearson  Product  Moment  Correlation  Coefficients 
and  Supporting  Data  for  Each  of  the  OAP  Primary  Factors 
on  Study  1  Test-Re test  Reliability  Analysis  (Five  Week  Interval) 


Factor 

Test 

Retest 

JL 

t 

r 

800 

X 

5.31 

5.64 

18 

-2.00 

.86*** 

(Skill 

SO 

1.37 

1.28 

Variety) 

se 

.51 

.48 

801 

X 

5.71 

5.76 

19 

-.26 

.76*** 

(Task 

SO 

1.38 

1.06 

Identity) 

Se 

.67 

.52 

802 

7 

6.16 

6.24 

19 

-.50 

.88*** 

(Task 

SO 

1.43 

1.34 

Significance) 

se 

.49 

.46 

804 

7 

5.21 

5.74 

19 

-2.16* 

.51* 

(Job 

SD 

1.16 

.98 

Feedback) 

se 

.81 

.69 

805 

7 

5.02 

5.11 

19 

-.45 

.78*** 

(Work 

SO 

1.27 

1.28 

Support) 

se 

.60 

.60 

806 

7 

6.04 

6.02 

19 

.10 

.72*** 

(Keed  for 

SD 

1.27 

1.27 

Enrichment) 

Se 

.67 

.67 

810 

7 

5.61 

5.08 

18 

2.40* 

.65** 

(Job  Per- 

SD 

.72 

1.23 

formance  Goals)Se 

.43 

.73 

811 

X 

5.97 

6.11 

19 

-.55 

.75*** 

(Pride) 

SO 

1.53 

1.36 

se 

.77 

.68 

812 

7 

5.58 

5.83 

18 

-1.64 

.81*** 

(Task 

SD 

1.08 

.96 

Character¬ 

istics) 

Se 

.47 

.42 
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Factor 

Test 

Retest 

JL 

JL 

r 

813  x 

4.83 

4.92 

19 

-.34 

.69*** 

(Task  SO 

1.56 

1.37 

Autonomy )  Se 

.87 

.76 

814  "x 

4.97 

5.12 

17 

-.79 

.87*** 

( Work  SO 

1.46 

1.49 

Repetition)  Se 

.53 

.54 

816  x 

2.44 

3.00 

18 

-2.05 

.72*** 

(Oesired  SO 

1.26 

1.65 

Repeti ti ve  Se 
Easy  Tasks) 

.67 

.87 

817  X 

5.19 

5.39 

19 

-.95 

.78*** 

(Advance-  SO 

1.39 

1.34 

ment/Recog-  Se 
nition) 

.65 

.63 

818  x 

5.20 

5.37 

18 

-.74 

.85*** 

(Management  SO 

1.73 

1.84 

Supervi$ion)Se 

.67 

.71 

819  X 

5.02 

5.22 

19 

-1.07 

.90*** 

(Supervi-  SO 

1.80 

1.90 

sory  Comm  Se 
Climate) 

.57 

.60 

820  X 

5.02 

5.38 

19 

-1.79 

.83*** 

(Organiza-  SD 

1.53 

1.43 

tional  Comm  Se 
Climate) 

.63 

.59 

821  X 

5.89 

6.19 

18 

-1.26 

.52* 

(Work  Group  SD 

.95 

1.09 

Effective-  Se 
ness) 

.66 

.76 

822  X 

6.05 

6.04 

17 

.09 

.96*** 

(Job  SD 

.95 

1.24 

Rel ated  Se 

Satisfaction) 

.19 

.25 

823  X 

5.60 

5.96 

15 

-1.59 

.76*** 

(Job  SD 

1.32 

1.27 

Related  Se 

Training) 

.65 

.63 

824  X 

5.38 

5.64 

19 

-1.51 

.88*** 

( General  SD 

1.52 

1.47 

Organiza-  Se 
tional  Climate 

.53 

.51 

★ 

P 

< 

.05 

x  = 

Mean 

kk 

p 

< 

.01 

SO  = 

Standard 

Deviation 

kirk 

p 

< 

.001 

Se  = 

Standard 

Error  of 
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Table  2 


Cronbach's  Alpha  Coefficients  and  Supporting  Data 
for  Each  of  the  OAP  Primary  Factors  (Study  1) 


Factor 


Test 

N  alpha  rho  ~ 


Retesl 
rho 


N  ""alpha 


n 


800  19  .67 

(Skill 

Variety) 


801  19  .82 

(Task 

Identity) 

802  19  .92 

(Task 

Significance) 


804  19  .77 

(Job 

Feedback) 

805  19  .66 

(Work 

Support) 

806  19  .95 

(Need  for 
Enrichment) 


810  18  .35 

(Job  Per¬ 
formance  Goals) 


811  19  .79 

(Pride) 


.52  2 

.72  2 

.86  2 

.62  2 

.41  3 

.81  5 

.17  5 

.68  2 


18  .82  .70  2 

19  .40  .25  2 

19  .90  .82  2 

19  .46  .30  2 

19  .73  .58  3 

19  .89  .65  5 

19  .77  .44  5 

19  .98  .97  2 


812  19  .86  .46  8  18  .86  .43  8 

(Task 

Character¬ 

istics) 


813  19  .88  .65  4  19  .87  .63  4 

(Task 

Autonomy) 
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J 


■IB 

■■■■■ 

- R5T55T - 

N 

aipna 

■55 

n 

N 

aipna 

FT- 

814 

(Work 

Repeti tion) 

17 

.72 

.62 

2 

19 

.71 

.56 

2 

816 

(Desired 

Repeti ti ve 

Easy  Tasks) 

18 

.49 

.37 

2 

19 

.63 

.46 

2 

817 

(Advance¬ 

ment/Recog¬ 

nition) 

19 

.87 

.57 

5 

1  9 

.90 

.65 

5 

818 

(Management 

Supervision) 

18 

.96 

.75 

8 

18 

.98 

.87 

8 

819 

(Supervi¬ 
sory  Comm 

Cl imate) 

19 

.96 

.75 

8 

19 

.98 

.86 

8 

820 

(Organiza¬ 
tional  Comm 

Cl imate) 

19 

.95 

.69 

9 

19 

.97 

.80 

9 

821 

(Work  Group 
Effective¬ 
ness) 

18 

.89 

.63 

5 

19 

.91 

.74 

5 

822 

(Job 

Rel ated 
Satisfaction) 

19 

.83 

.42 

7 

17 

.90 

.57 

7 

823 

(Job 

Rel ated 
Training) 

16 

.91 

.84 

2 

18 

.93 

.87 

2 

824 

(General 
Organiza¬ 
tional  Climate) 

19 

.96 

.72 

10 

19 

.97 

.80 

10 

N  =  Number  of  subjects 

n  =  Number  of  items 

rho  =  Average  item  intercorrelation 
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additional  criteria  that  a  ^‘rference  between  alpha  coefficients  of  more 
than  plus  or  minus  .10  could  indicate  problems,  the  same  six  factors  were 
again  identified.  In  addition,  Factor  811  (Pride)  also  showed  some  incon¬ 
sistency  from  test  to  retest. 

As  a  further  standard  of  comparison.  Table  3  presents  the  coefficient 
alpha  calculations  for  the  entire  pretest  (data  collection  prior  to  consult¬ 
ing  evaluation)  data  base.  Results  are  consistent  with  Study  1,  as  Factor 
801  (Task  Identity),  Factor  804  (Job  Feedback)  and  Factor  805  (Work  Support) 
were  again  below  .70. 

A  comment  should  be  made  about  the  results  of  Cronbach's  alpha  analysis 
in  Study  1.  In  all  but  one  case  where  problems  with  low  alpha  coefficients 
were  noted,  the  problems  seemed  related  to  a  low  number  of  items  in  a  fac¬ 
tor.  In  four  of  the  six  cases,  factors  in  question  had  two  items  per  fac¬ 
tor.  One  other  factor  had  three  items.  The  only  exception  here  was  Factor 
810.  Further  inspection  of  the  alpha  analysis,  however,  showed  that  one 
item  was  poorly  correlated  with  the  other  items  in  the  factor.  Without  this 
item,  the  alpha  coefficient  for  Factor  810  was  a  respectable  .77  on  the  test 
and  .84  on  the  retest. 
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Table  3 


Cronbach's  Alpha  Coefficients  and  Supporting  Data 
for  Each  of  the  OAP  Primary  Factors  for  the 
Entire  Pretest  Data  Base 


Factor 

MM9IH9HI 

wmommmm 

EHSHOZSHBHHI 

n 

800 

(Skill 

Variety) 

55,539 

.81 

.68 

2 

801 

(Task 

Identity) 

55,276 

.58 

.41 

2 

802 

(Task 

Signi ficance) 

55,897 

.79 

.65 

2 

804 

(Job 

Feedback) 

55,788 

.66 

.49 

2 

805 

(Work 

Support) 

54,103 

.41 

.20 

3 

806 

(Need  for 
Enrichment) 

55,058 

.90 

.64 

5 

810 

(Job  Per¬ 
formance  Goals 

54,959 

.72 

.34 

5 

811 

(Pride) 

55,651 

.90 

.81 

2 

812 

(Task 

Character- 
i sties) 

54,123 

.84 

.39 

8 

813 

(Task 

Autonomy) 

54,848 

.85 

.59 

4 
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814  55,526  .70  .54  2 

(Work 

Repetition) 

816  54,798  .70  .53  2 

(Desired 

Repeti ti ve 
Easy  Tasks) 

817  54,085  .78  .41  5 

( Advance¬ 
ment/  Recog¬ 
nition) 

818  54,060  .94  .67  8 

(Management 

Supervision) 

819  53,450  .95  .68  8 

(Supervi¬ 
sory  Comm 

Cl Imate) 

820  54,613  .92  .57  9 

(Organiza¬ 
tional  Comm 

Climate) 

821  54,261  .87  .57  5 

(Work  Group 

Effective¬ 

ness) 

822  49,374  .84  .42  7 

(Job 

Related 

Satisfaction) 

823  50,630  .73  .58  2 

(Job 

Rel a ted 
Training) 

824  54,805  .92  .55  10 

(General 

Organiza¬ 
tional  Climate) 


N  *  Number  of  subjects 

n  *  Number  of  Items 

rho  *  Average  Item  Intercorrelation 
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Study  2 


The  results  of  the  second  study  may  be  seen  in  Tables  4  and  5.  As  seen 
in  Table  4,  all  but  four  of  the  retest  coefficients  were  significant  at 
p  <  .001  level  or  better.  The  exceptions  were  Factor  806  (Need  for  Enrich¬ 
ment),  Factor  814  (Work  Repetition),  Factor  816  (Desired  Repetitive  Easy 
Tasks)  and  Factor  823  (Job  Related  Training).  Factor  823  was  significant 
precisely  at  the  .001  level.  Factor  802  C'ask  S*  g  'r: ;  ■-’l  showed  a  signif¬ 

icant  decrease  from  test  to  retest.  In  additic.:  a  the  „ot relation  coeffi¬ 
cients,  Table  4  shows  the  means,  standard  deviations,  and  standard  errors  of 
measurement  for  both  test  and  retest,  the  sample  sire  for  each  factor  and 
results  of  correlated  means  t-tests  (DuBois, 

As  expected,  the  retest  correlation  coefficients  were  lower  for  Study  2 
than  for  Study  1  (see  Table  1).  The  lower  coefficients  also  caused  higher 
standard  errors  indicating  a  degree  of  measurement  "wobble"  over  the  six 
month  interval.  While  these  results  were  expected,  effort  will  be  made  to 
strengthen  factor  stability  and,  therefore,  lower  standard  errors  of  revised 
OAP  factors.  It  should  also  be  noted  that  the  Factors  remaining  most  stable 
included  measurements  of  total  organizational  climate  and  measurements  deal¬ 
ing  with  the  job  itself.  The  least  stable  factors  for  the  six  month  inter¬ 
val  dealt  with  goals  and  with  training. 

Table  5  shows  the  results  of  the  Cronbach’s  alpha  analysis  on  the  same 
data.  The  sample  size,  coefficient  alpha,  average  interitem  correlation  and 
number  of  items  per  factor  are  reported  for  both  test  and  retest  conditions. 
Based  on  the  criteria  set  forth  in  the  methodology  section,  six  factors 
showed  a  coefficient  below  .70  on  either  test  o-  retest.  These  were  Factor 
801  (Task  Identity),  Factor  804  (Job  Feedback'1  :"crpr  805  (work  Support), 
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Table  4 


Pearson 

Product 

Moment  Correlation 

Coefficients 

and 

Supporting  Data 

for  Each  of 

the  OAP 

Primary  Factors 

on  Study  2  Test- 

-Retest 

Rel  i  ab  i  1  i  ty 

Analysi s 

(Six  Month 

Interval 

Factor 

Test 

Retest 

JL 

t 

r 

800 

X 

3.33 

3.29 

380 

.44 

.44* 

(Skill 

SD 

1.56 

1.61 

Variety) 

Se 

1.17 

1.20 

801 

7 

4.14 

3.97 

353 

1.76 

.33* 

(Task 

SD 

1.50 

1.55 

Identi ty) 

Se 

1.23 

1.27 

802 

7 

4.95 

4.62 

391 

3.39* 

.34* 

(Task 

SO 

1.65 

1.72 

Signi ficance) 

Se 

1.34 

1.40 

804 

7 

3.90 

3.73 

379 

1.74 

.25* 

(Job 

SD 

1.47 

1.48 

Feedback) 

se 

1.27 

1.28 

805 

7 

4.33 

4.10 

359 

3.09 

.26* 

(Work 

SD 

1.18 

1.11 

Support) 

se 

1.02 

.95 

806 

7 

5.04 

4.73 

373 

2.85 

.14 

(Need  for 

sn 

1.53 

1.68 

Enrichment) 

se 

1.42 

1.56 

810 

7 

4.34 

4.17 

363 

2.31 

.21* 

(Job  Per- 

SD 

1.05 

1.10 

formance  Goals)Se 

.93 

.98 

811 

X 

4.04 

3.71 

382 

3.27 

.43* 

(Pride) 

SD 

1.90 

1.81 

Se 

1.43 

1.37 

812 

7 

4.20 

3.96 

331 

3.24 

.43* 

(Task 

SD 

1.22 

1.32 

Character- 

Se 

.92 

.97 

istics) 
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Factor 

Test 

Retest 

H 

t 

r 

813  X 

2.86 

2.81 

359 

-.70 

.56* 

(Task  SD 

1.43 

1.41 

Autonomy)  Se 

.95 

.94 

814  X 

5.53 

5.44 

388 

1.02 

.13 

( Work  SD 

1.34 

1.48 

Repetition)  Se 

1.25 

1.38 

816  X 

3.43 

3.43 

385 

-.04 

.16 

(Desired  SD 

1.57 

1.53 

Repetitive  $e 
Easy  Tasks) 

1.44 

1.40 

817  X 

3.83 

3.69 

380 

2.03 

.39* 

(Advance-  SO 

1.19 

1.21 

men t/ Recog-  Se 
nition) 

.93 

.95 

818  X 

4.82 

4.72 

362 

.98 

.20* 

(Management  SD 

1.56 

1.61 

Supervision)$e 

1.40 

1.44 

819  X 

4.39 

4.36 

If! 

,?0 

.26* 

(Supervi-  SD 

1.61 

1.60 

sory  Comm  Se 

Cl imate) 

1.38 

1.38 

820  X 

3.78 

3.73 

362 

.68 

.43* 

(Organiza-  SD 

1.35 

1.44 

tional  Comm  Sp 
Climate) 

1.02 

1.09 

821  X 

5.02 

4.90 

362 

1.40 

.30* 

(Work  Group  SD 

1.34 

1.49 

Effective-  Se 
ness) 

1.12 

1.25 

822  X 

4.16 

3.97 

334 

2.33 

.43* 

(Job  SD 

1.41 

1.42 

Rel ated  Se 

Satisfaction) 

1.06 

1.07 

823  X 

3.97 

3.93 

369 

.44 

.18+ 

(Job  SD 

1.64 

1.62 

Rel ated  Se 

Training) 

1.49 

1.47 

824  X 

3.83 

3.67 

356 

1.87 

.42* 

(General  SD 

1.47 

1.47 

Organiza-  Se 
tional  Cl imate 

1.12 

1.12 

*  p  <  .001 

x  =  Mean 

+  significant  at 

SD  =  Standard  Deviation 

the  .001  level, 

Se  =  Standard  Error  of  Measurement 

but  not  beyond 
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Factor  810  (Job  Performance  Goals),  Factor  814  (Work  Repetition)  and  Factor 
816  (Desired  Repetitive  Easy  Tasks).  Under  the  additional  criteria  that  a 
difference  between  alpha  coefficients  of  more  than  plus  or  minus  .10  would 
indicate  problems,  two  of  these  same  five  factors  (801  and  814)  again  were 
indicated.  These  results  may  be  compared  with  results  from  the  entire  pre¬ 
test  data  base  by  contrasting  Tables  5  and  3. 

As  was  the  case  in  Study  1,  problems  with  low  alpha  coefficients  seemed 
related  to  low  number  of  items  in  a  factor.  In  four  of  the  six  cases,  fac¬ 
tors  In  question  had  two  items  per  factor.  One  other  factor  had  three 
items.  Again,  the  only  exception  was  Factor  810,  a  factor  containing  one 
item  poorly  correlated  with  the  other  items  in  the  factor.  Without  this 
item,  the  alpha  coefficient  for  Factor  810  was  .79  on  the  test  and  .82  on 
the  retest. 
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Table  5 


Cronbach's  Alpha  Coefficients  and  Supporting  Data 
for  Each  of  the  OAP  Primary  Factors  (Study  2) 


Factor 

Test 

Retest 

N 

alpha 

BiEi 

n 

N 

alpha 

n 

800 

(Skill 

Variety) 

389 

.82 

.70 

2 

388 

.88 

.78 

2 

801 

(Task 

Identity) 

367 

.59 

.41 

2 

383 

.65 

.48 

2 

802 

(Task 

Significance) 

394 

.73 

.58 

2 

395 

.80 

.66 

2 

804 

(Job 

Feedback) 

386 

.69 

.52 

2 

.  r  j 

.57 

2 

805 

(Work 

Support) 

375 

.44 

.21 

3 

380 

.37 

.18 

3 

806 

(Need  for 
Enrichment) 

388 

.91 

.68 

5 

383 

.93 

.74 

5 

810  376 

(Job  Per¬ 
formance  Goals) 

.69 

.30 

5 

385 

.73 

.35 

5 

811 

(Pride) 

390 

.91 

.84 

2 

391 

.88 

.79 

2 

812 

(Task 

Character¬ 

istics) 

353 

.86 

.45 

8 

372 

.88 

.49 

8 

813 

(Task 

Autonomy) 

377 

.84 

.58 

4 

380 

.86 

.61 

4 
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factor 

Test 

Retest 

N 

alpha 

~rfio 

n 

N 

aipna 

rno 

n 

814 

(Work 

Repeti tion) 

393 

.64 

.47 

2 

394 

.80 

.66 

2 

816 

(Desired 

Repetitive 

Easy  Tasks) 

387 

.68 

.51 

2 

397 

.66 

.49 

2 

817 

(Advance¬ 

ment/Recog¬ 

nition) 

389 

.79 

.43 

5 

390 

.81 

.47 

5 

818 

(Management 

Supervision) 

374 

.95 

.69 

8 

387 

.95 

.70 

8 

819 

(Supervi¬ 
sory  Comm 

Cl imate) 

375 

.94 

.68 

8 

382 

.95 

.72 

8 

820 

(Organiza¬ 
tional  Cornn 

Cl imate) 

380 

.91 

.55 

9 

379 

.93 

.62 

9 

821 

(Work  Group 
Effective¬ 
ness) 

375 

.84 

.52 

5 

384 

.88 

.60 

5 

822 

(Job 

Related 

Satisfaction) 

359 

.86 

.46 

7 

374 

.87 

.48 

7 

823 

(Job 

Rel ated 
Training) 

378 

.74 

.59 

2 

388 

.77 

.63 

2 

824 

(General 
Organiza¬ 
tional  Climate) 

377 

i 

.92 

.53 

10 

377 

.93 

.58 

10 

I 


N  =  Number  of  subjects 

n  =  Number  of  Items 

rho  =  Average  Item  Intercorrelation 
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Discussion,  Conclusions,  and  Recommendations 


Discussion 


Perhaps  the  simplest  way  to  integrate  and  discuss  results  is  to  consider 
each  factor  that  had  a  reliability  coefficient  below  the  minimum  acceptable 
range.  In  so  doing,  the  following  code  will  be  used  tc  refer  specific  parts 
of  the  report: 


TRl  -  Test-Retest  {Study  1,  Table  1) 

Aj  -  Cronbach's  alpha  (Study  1 ,  Tab1'5 

-  Cronbach's  alpha  on  the  entire  pretest  data  base 
(Study  1,  Table  3) 

TR2  -  Test-Retest  (Study  2,  Table  4) 

A2  -  Cronbach's  alpha  (Study  2,  Table  5) 

Factor  800,  Skill  Variety  (Aj):  This  factor  had  an  alpha  of  less  than 
.67  for  the  first  (test)  administration  in  Study  1,  but  the  coefficient  for 
the  retest  was  exactly  .70.  All  other  studies  showed  coefficients  in  the 
acceptable  range.  This  is  a  two  Item  factor. 

Factor  801,  Task  Identity  (Aj,  A^.  fy)'-  This  factor  remained 
reasonably  consistent  on  the  test-retest  coefficients,  but  showed  a  great 
deal  of  variation  on  the  alpha  coefficients,  with  some  results  being  quite 
low.  The  Ajjb  value  was  .58  which  seems  a  mi  jromd  for  coefficients 

ranging  from  .40  to  .82.  This  is  a  two  item  factor 


Factor  80* ,  J  .b  Feedback  ( TR^ ,  Alt  Adb,  A2):  This  factor  was 

below  the  acceptable  level  on  both  parts  of  Study  1  and  on  calculations  from 
the  data  base.  In  Study  2,  the  cest-retest  was  an  acceptable  .25,  but  alpha 
levels  were  low  to  marginal  (.5b  and  .73).  This  is  a  two  item  factor. 

Factor  805,  Work  Support  (Aj,  Adb,  A2):  This  factor  was  below 
acceptable  levels  on  all  Cronbach's  alpha  calculations,  with  very  low  coef¬ 
ficients  in  Study  2  (.44  and  .37).  Test-retest  coefficients  were  acceptable 
in  terms  of  statistical  significance.  This  is  a  three  item  factor. 

Factor  806,  Need  for  Enrichment  (TR2):  This  factor  was  acceptable  on 
the  test-retest  part  of  Study  1,  but  showed  a  very  poor  correlation  in  Study 
2  (.14).  Despite  this,  however,  all  alpha  coefficients  were  quite  high, 
varying  from  .89  to  .95  on  Studies  1  and  2  and  .90  for  the  data  base.  A 
comment  will  be  made  about  this  factor  below.  This  is  a  five  item  factor. 

Factor  310,  7. oh  Periormat.ee  Coa--.  ;  A^ ,  A2):  This  factor 

showed  considerable  variability.  Especially  noteworthy  were  alpha  coeffi¬ 
cients  ranging  from  .35  to  .77.  Despite  this,  however,  the  alpha  coeffi¬ 
cient  for  the  entire  data  base  was  acceptable  (.72),  suggesting  that 
extremely  large  camp'. r  sii.e  may  mask  some  of  the  lack  of  stability  and 
internal  consistency  of  this  factor.  Nevertheless,  interitem  correlations 
must  be  improved.  In  particular,  one  item  must  be  revised  or  removed.  This 
is  a  five  item  factor. 
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Factor  811,  Pride  ( ) :  This  factor  showed  a  shift  in  value  of  alpha 
coefficients  from  .79  to  .98  on  Study  1.  Despite  the  shift,  all  coeffi¬ 
cients  were  good  to  excellent,  with  the  entire  data  base  alpha  being  .90. 
It  seems  likely  this  variation  was  sample  specific,  but  results  do  reinforce 
the  high  probability  of  variability  in  factors  with  few  items.  This  is  a 
two  item  factor. 

Factor  814,  Work  Repetition  (TR2 »  This  factor  was  barely  with¬ 

in  the  minimum  acceptable  standards  on  Cronbach's  alpha  for  Study  1  (.72  and 
.71)  and  for  the  data  base  (.70).  On  Study  2,  however,  the  factor  showed  a 
distinct  lack  of  test-retest  stability  (.13).  while  'Vor-rirn  slightly  on  the 
coefficient  alpha  (.64  and  .80).  Results  again  point  out  the  instability  of 
a  two  item  factor. 

Factor  816,  Desired  Repetitive  Easy  Tasks  (A^,  TR2,  A2):  Results 

for  this  factor  are  almost  a  mirror  image  of  Factor  814.  Test-retest  sta¬ 
bility  for  Study  1  was  acceptable  (.72);  alpha  coefficients  were  low  (.49 
and  .63)  on  Study  1  and  barely  acceptable  (.70)  for  the  data  base.  For  the 
longer  interval,  however,  stability  dropped  sharply  (.16)  while  alpha  coef¬ 
ficients  remained  below  standard  (.68  and  .66).  This  is  a  two  item  factor. 

Factor  821,  Work  Group  Effectiveness  (TRj):  This  factor  had  compara¬ 
tively  low  stability  on  Study  1.  Alpha  coefficients  were  good  throughout, 
however,  ranging  from  .86  to  .91.  The  stability  coefficient  for  the  six 
month  Interval  was  acceptable  and  in  line  with  oth^r  tutors.  This  factor 


Is  likely  unique  in  one  respect:  respondents  seem  more  likely  to  give  posi¬ 
tive  responses  to  this  factor  than  any  other.  This  tendency  to  "fake  good" 
(sometimes  called  social  desirability)  may  well  be  the  cause  of  sample  spe¬ 
cific  variations  iri  the  factor,  or,  for  that  matter,  any  of  the  factors. 
Bias  in  the  form  of  "making  myself  look  good,"  therefore,  is  an  issue  that 
needs  investigation  for  all  OAP  items  and  factors. 

Factor  823,  Job  Related  Training  ( TR2 ) :  Test-retest  stability  was 
good  (.76),  and  alpha  coefficients  were  quite  high  (.91  and  .93)  for  this 
factor  on  Study  1.  The  alpha  coefficient  for  the  entire  data  base  dropped, 
however,  to  .73,  a  figure  barely  in  the  acceptable  range.  This  pattern  con¬ 
tinued  in  the  alpha  coefficients  for  Study  2  (.74  and  .77),  but  the  test- 
retest  coefficient  on  Study  2  dropped  sharply  (.18).  This  appears  to  be 
mainly  the  result  of  so  few  items  in  the  factor.  Factor  823  is  a  two  item 
factor. 

Two  Final  Comments.  First,  a  comment  about  the  findings  regarding  Fac¬ 
tor  806  is  in  order.  It  will  be  recalled  this  factor  showed  problems  only 
with  long-range  stability,  not  with  short-range  stability  or  internal  con¬ 
sistency.  It  seems  highly  likely  that  this  facto>'  taps  perceptions  of 
issues  which  are  likely  to  change  over  time.  This  change  in  people  and 
conditions  should  not  be  confused  with  measurement  error.  Nunnally  and 


Durham  (1975)  point  out  that  if  retests  "...given  six  months  apart  correlate 
less  than  those  given  two  weeks  apart,  in  a  sense  the  difference  Is  not 
because  of  'error,'  but  because  of  systematic  changes  in  people"  (p.336). 
It  appears  quite  possible  this  is  the  case  for  the  factor  in  question. 

Second,  in  contrasting  the  two  methods  of  assessing  reliability, 
Cronbach's  alpha  is  generally  held  in  higher  regard  than  the  test- retest 
method  (see,  for  example,  Carmines  and  Zeller,  1979).  Nevertheless,  stabil¬ 
ity  and  dependability  of  factor  scores  over  time  is  an  important  issue, 
especially  in  terms  of  evaluation  of  consulting  effectiveness.  While  alpha 
coefficients  should  probably  be  considered  more  important,  final  interpreta¬ 
tions  and  judgements  should  be  based  on  careful  weighing  of  both  stability 
and  internal  consistency  estimates  of  OAP  reliability. 

Conclusions  and  Recommendations 

Looking  at  the  results  in  retrospect,  it  is  possible  to  say  first  that 
the  OAP  generally  shows  acceptable  to  excellent  reliability.  Factor  differ¬ 
ences  have  been  noted,  and,  clearly,  some  factors  are  stronger  than  others. 
Nevertheless,  the  instrument  seems  quite  reliable  enough  to  provide  a  source 
of  real  time  Air  Force  systemic  data.  In  terms  of  its  use  as  a  consulting 
tool,  results  must  be  considered  on  a  factor  to  factor  basis,  especially  as 
the  results  may  Impact  consulting  evaluation  methodology. 

Second,  even  for  factors  that  show  weaknesses  in  reliability,  remedial 
actions  seem  fairly  obvious.  It  is  clear,  for  example,  that  most  of  the 
Instability  and  low  internal  consistency  can  be  attributed  to  factors  with 
few  Items.  This  is  not  surprising,  since  both  types  of  reliability  can  be 
Increased  by  adding  items  to  a  scale  or  factor.  Not  just  any  items  will  do. 


however-,  as  the  .  nee  «i  in  :  a '.tor  810  'hows.  One  item  with  low  inter¬ 

item  correlations  can  severely  damage  overall  factor  reliability,  whether 
stability  or  internal  cons i storey .  It  seems  important,  then,  to  center 
revision  efforts  on  attempts  to  combine  an  optimum  number  of  six  to  eight 
highly  intercorrol ated  i terns  into  fewer,  more  precise  factors.  In  particu¬ 
lar,  the  current  two  or  three  item  factors  should  be  expanded  or  combined 
into  more  efficient  and  effective  measures. 

Third,  the  issue  of  positive  response  bias  as  related  to  the  OAP  should 
be  studied.  The  possible  existence  of  this  problem  was  pointed  out  in  rela¬ 
tion  to  Factor  821.  The  likelihood  is,  however,  that  this  is  a  form  of  bias 
in  all  factors.  In  a  situation  where  outside  "experts"  are  looking  at  an 
organization,  the  tendency  to  make  supervisor  and  organization  look  good  can 
be  very  strong.  Demonstrating  that  results  are  not  unduely  influenced  by 
such  tendencies  has  important  implications  for  both  reliability  and  validity 
of  the  OAP.  Fortunately,  the  issue  is  not  a  complex  one  to  deal  with,  and 
at  least  three  options  are  available  for  such  a  study  (Crowne  &  Marlowe, 
1960;  Green  wa":  d  ..  :ow ,  j  i  c,  >s  .  ••,  Hi  c.  .u,  n  Ca*  dascia,  1978). 

Fourth,  consideration  of  reliability  studies  should  be  included  when 
selecting  a  factor  analysis  model  for  the  OAP  revision.  Conlon  (1980)  has 
demonstrated  what  may  be  a  nee  parsimonious  OAP  facto1"  structure.  Whether 
or  not  these  are  the  f-’na".  revised  factors,  However,  the  model  of  factor 
analysis  is  an  important  consideration.  Coefficient  theta,  for  example,  is 
based  on  the  model  of  principle  components,  while  coefficient  omega  is  based 
on  the  common  factor  analysis  model  (Carmines  and  Zeller,  1979).  Both  have 
advantages,  disadvantages  and  different  uses.  Thus,  the  model  of  factor 
analysis  used  for  any  revision  may  play  a  part  in  not  only  the  extent  to 
which  factor  reliability  is  possible,  but  also  on  how  reliability  is  deter¬ 
mined  and  measured. 
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In  accordance  with  paragraph  30,  APR  12-35,  The  Air  Force  Privacy  Act  • 
Program,  the  following  information  about  this  survey  is  provided: 

a.  Authority:  10  O.S.C- ,  3012,  Secretary  of  the  Air  Force:  Powers  and 
Duties,  Delegation  by  Compensation  E.  0.  9397,  22  Nov  43,  Numbering  System 
for  Federal  Accounts  Relating  to  Individual  Persons. 

b.  Principal  Purpose:  The  survey  is  being  conducted  to  assess  your 
organization  from  a  leader  din  and  nanaoement  perspective. 

c.  Routi ne  Uses  :  Infomai  icr  provided  by  respondents  will  be  treated 
confidential ly.  The  averaged  data  will  be  used  for  organizational  strength 
and  weakness  identification  and  Air  Force  wide  research  and  development 
purposes. 

d.  Participation:  Response  to  this  survey  is  voluntary.  Your  coopera¬ 
tion  in  this  effort  is  appreciated. 


r 


[PLEASE  DO  NOT  TEAR,  MARK  ON,  OR  OTHERWISE  DAMAGE  THIS  BOOKLET] 
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EXPIRATION  DATE:  31  Oct  1981 


SCN  81-  14 

GENERAL  INFORMATION 


The  leaders  of  your  organization  are  genuinely  interested  in  improving  the 
overall  conditions  within  their  areas  of  responsibility.  Providing  a  more 
satisfying  Air  Force  way  of  life  and  increasing  organizational  effectiveness 
are  also  goals.  One  method  of  reaching  these  goals  is  by  continual  refine¬ 
ment  of  the  management  processes  of  the  Air  Force.  Areas  of  concern  include 
job  related  issues  such  as  leadership  and  management;  training  and  utiliza¬ 
tion;  motivation  of  and  concern  for  people;  and  the  communication  process. 

This  survey  is  intended  to  provide  a  means  of  identify! ng  areas  within  your 
organization  needing  the  greatest  emphasis  in  the  immediate  future.  You 
will  be  asked  questions  about  your  job,  work  group,  supervisor,  and  organi¬ 
zation.  For  the  results  to  be  useful,  it  is  important  that  you  respond  to 
each  statement  thoughtfully,  honestly,  and  as  frankly  as  possible.  Remem¬ 
ber,  this  is  not  a  test,  there  are  no  right  or  wrong  responses. 

Your  completed  response  sheet  will  be  processed  by  automated  eqiupment,  and 
be  summarized  in  statistical  form.  Your  individual  response  will  remain 
confidential,  as  it  will  be  combined  with  the  responses  of  many  other  per¬ 
sons,  and  used  for  organizational  feedback  and  possibly  Air  Force  wide 
studies. 


KEY  WORDS 

The  following  should  be  considered  as  key  words  throughout  the  survey: 

—  Supervisor:  The  person  to  whom  you  report  directly. 

--  Work  Group:  All  persons  who  report  to  the  same  supervisor  that  you 
do. 

--  Organization:  Your  squadron.  However,  if  you  work  in  staff/support 
agencies,  the  division  or  directorate  would  be  your 
organi zation. 
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INSTRUCTIONS 


1.  All  statements  may  be  answered  by  filling  in  the  appropriate  spaces  on 

the  response  sheet  provided.  If  you  do  not  find  a  response  that  fits  your 

case  exactly,  use  the  one  that  is  the  closest  to  the  way  you  feel. 

2.  Be  lure  that  you  have  completed  Section  1  of  the  response  sheet,  as 

instructed  by  the  survey  administrator,  before  beginning  Section  2. 

3.  Please  use  the  pencil  provided,  and  observe  the  following: 

--Make  heavy  black  marks  that  fill  the  spaces. 

--Erase  cleanly  any  responses  you  wish  to  change. 

— Make  no  stray  markings  of  any  kind  on  the  response  sheet. 

--Do  not  staple,  fold  or  tear  the  response  sheet. 

--Do  not  make  any  markings  on  the  survey  booklet. 

4.  The  response  sheet  has  a  0-7  scale.  The  survey  statemerts  normally 
require  a  1-7  response.  Use  the  zero  (0)  response  only  if  tne  statement 
truly  does  not  apply  tc  your  situation.  Statements  are  responded  to  by 
marking  the  appropriate  space  on  the  response  sheet  as  in  the  following 
example: 

Using  the  scale  below,  evaluate  the  sample  statement. 

1  =  Strongly  disagree  5  =  Slightly  agree  ^ 

2  =  Moderately  disagree  6  =  Moderately  agree 

3  =  Slightly  disagree  7  =  Strongly  agree 

4  =  Neither  agree  nor  disagree 

Sample  Statement.  The  information  your  work  group  receives  from  other  work 
groups  is  helpful. 

If  you  moderately  agree  with  the  sample  statement,  you  would  blacken  the 
oval  (6)  on  the  response  sheet. 

NA 

Sample  Response:  (0)  (1)  (2)  (3)  (4)  (5)  (6)  (7) 


5.  When  you  have  completed  the  survey,  please  turn  in  the  survey  materials 
as  instructed  in  the  introduction. 


ii  i 


35/36 


uACkdROUND  INFORMATION 


This  sect i or.  of  the  ,r. —  v  background.  The  information 

requested  is  to  -Vs^t  :i  .  '  1  ;•  .  ..  belong  to  are  accurately 

represented  jnd  no*  to  ■  M  ..  '• j  i  ■*<.  ndi vidua! .  Please  use  the 
separate  respor-s?  sheet  ,:rd  ua>kr  t  ip  ova’  which  corresponds  to  your 
response  to  each  quest.  i~r. 

1.  Total  years  it  the  A, r  fo"ce- 

1.  Less  than  1  year. 

2.  More  than  1  yec*  ,  loss  than  2  years. 

3.  More  than  2  years,  less  than  3  years. 

4.  More  than  3  years,  less  than  4  years. 

5.  More  than  4  years,  less  than  8  years. 

6.  More  than  8  years,  less  than  12  years. 

7.  More  the..'  12  ye-. ,  .. . 

2.  Total  months  in  present  ta. ecr  field: 

1.  Less  than  1  nonth. 

2.  More  than  1  month,  less  than  6  months. 

3.  More  than  6  months,  less  than  12  months. 

4.  More  than  ’2  rv  12  .  ,  ’  e. ,  '•  ••  1 months. 

5.  More  than-  12  cc;/-1.-,  t.u.i  24  months. 

6.  More  than  24  months,  less  than  36  months. 

7.  More  than  36  months. 

3.  Tof  V  r,r  •  *  .  .  *  -  • 

1.  less  tna.,  1  month. 

2.  More  than  1  month,  less  than  6  months. 

3.  More  than  6  months,  less  than  12  months. 

4.  More  than  i?  months,  less  than  months.  0 

5.  More  than  id  months,  less  Loan  24  months. 

6.  Mor.-  tVm  ?4  <«  le^'  ''M-  3P  nnot,.«- . 

7.  More  .Mo.  26  mo  .  •... 

4.  Total  months  in  present  position: 

1.  Less  than  1  month. 

2.  More  than  1  month,  less  than  6  months. 

3.  More  than  6  months,  less  than  12  months. 

4.  More  than  12  months,  less  than  18  months. 

5.  More  than  18  months,  less  than  24  months. 

6.  More  than  24  months,  less  than  36  months. 

7.  More  than  36  months. 
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5.  Your  Ethnic  Group  is: 


1.  American  Indian  or  Alaskan  Native 

2.  Asian  or  Pacific  Islander 

3.  Black,  not  of  Hispanic  Origin 

4.  Hispanic 

5.  White,  not  of  Hispanic  Origin 

6.  Other 

6.  Your  highest  education  level  obtained  is: 

1.  Non-high  school  graduate 

2.  High  school  graduate  or  GED 

3.  Less  than  two  years  college 

4.  Two  years  or  more  college 

5.  Bachelors  Degree 

6.  Masters  Degree 

7.  Doctoral  Degree 

7.  Highest  level  of  professional  military  education  (residence  or 
correspondence) : 

0.  None  or  not  applicable 

1.  NCO  Orientation  Course  or  USAF  Supervisor  Course  (NCO  Phase  1  or  2) 

2.  NCO  Leadership  School  (NCO  Phase  3) 

3.  NCO  Academy  (NCO  Phase  4) 

4.  Senior  NCO  Academy  (NCO  Phase  5) 

5.  Squadron  Officer  School 

6.  Intermediate  Service  School  (i.e.,  ACSC,  AFSC) 

7.  Senior  Service  School  (i.e.,  AWC,  ICAF,  NWC) 

8.  How  many  people  do  you  directly  supervise? 

1.  None  5.  4  to  5 

2.  1  6.  6  to  8 

3.  2  7.  9  or  more 

4.  3 

9.  For  how  many  people  do  you  write  performance  reports? 

1..  None  5.  4  to  5 

2.  1  6.  6  to  8 

3.  2  7.  9  or  more 

4.  3 

10.  Does  your  supervisor  actually  write  your  performance  reports? 

1.  yes  2.  no  3.  not  sure 
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11.  Wmon  u!  uie  folio;  ::g  oeat  describes  your  marital  status? 


0. 

hot  ’13!  ri 

ed 

1. 

Mu  rr :au- 

Spc>: si  :  a  ci  / 

i  1  *  a  n 

employed  outside 

home. 

2. 

Married: 

Spouse-  is  a  civ 

i  ’  i  an 

employed  outside 

home- 

uucgraphi 

roily  . 

3. 

Marne:.: 

Spouse  ;:.3'  ej :p • 

pyed 

tutside  home. 

4. 

Married: 

Spo  j  "  not  •  "i  ip , 

'-/ed 

outside  home- 

cjographica  i  ly  -.epa. .  :ed. 

5.  Married:  Spouse  is  a  military  member. 

6.  Married:  Soouse  is  a  military  member-geographical ly  separated. 

7.  Single  Parer:.. 

12.  What  is  your  usual  work  schedule? 

1.  Day  shift,  normally  stable  hours. 

2.  Swing  shift  (about  1600-2400) 

3.  Mid  shift  (aNv.f 

4.  Rotating  ah  i srherlt'- 

5.  Day  or  shift  work  wit*-,  i n'-eg.-l ar/unstabl e  hours. 

6.  Freauent  TDY/travel  or  frequently  on-call  to  report  to  work. 

7.  Crew  schedule. 

13.  How  often  does  your  supervisor  hold  group  meetings? 

1.  Never  a.  Weekly 

2.  Occasionally  5.  Daily 

3.  Monthly  6.  Continuously 

14.  How  '/ftc  .  irs  gr  >  ,  .*»>.•••;  •:  ■.<><  -e  solve  nr  oh ;  ems  and  establish  goals? 

1.  Nev  s  Auo.i!.  half  the  time 

2.  Occasionally  4.  All  of  the  time 

15.  What  is  your  aeronautical  rating  and  current  status?  r 

1.  Nonrated,  not  on  ai rc  3.  Rat  <-•*<.  ir.  •rew/operations  job 

'  ■>  .  i,  .r.  w  :  list.  2a  '  -u'.  if  support  job 

16.  Which  of  the  following  best  describes  your  career  or  employment 
intentions? 

1.  Planning  to  retire  in  the  next  12  months 

2.  Will  continue  in/with  the  Air  Force  as  a  career 

3.  Will  most  likely  continue  in/with  the  Air  Force  as  a  career 

4.  May  continue  in/with  the  Air  Force 

5.  Will  most  likely  not  make  the  Air  Force  a  career 

6.  Will  separate/terminate  from  the  Air  Force  as  soon  as  possible 
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JOB  INVENTORY 


Below  are  items  which  relate  to  your  job.  Read  each  statement  carefully  and 
then  decide  to  what  extent  the  statement  is  true  of  your  job.  Indicate  the 
extent  to  which  the  statement  is  true  for  your  job  by  choosing  the  phrase 
which  best  represents  your  job. 

1  =  Not  at  al 1 

2  =  To  a  very  little  extent 

3  =  To  a  little  extent 

4  =  To  a  moderate  extent 

Select  the  corresponding  number  for  each  question  and  enter  it  on  the 
separate  response  sheet. 


5  =  To  a  fairly  large  extent 

6  =  To  a  great  extent 

7  =  To  a  very  great  extent 


17.  To  what  extent  does  your  job  require  you  to  do  many  different  things, 
using  a  variety  of  your  talents  and  skills? 

18.  To  what  extent  does  your  job  involve  doing  a  whole  task  or  unit  of  work? 

19.  To  what  extent  is  your  job  significant,  in  that  it  affects  others  in 
some  important  way? 

20.  To  what  extent  does  your  job  provide  a  great  deal  of  freedom  and  inde¬ 
pendence  in  scheduling  your  work? 

21.  To  what  extent  does  your  job  provide  a  great  deal  of  freedom  and  inde¬ 
pendence  in  selecting  your  own  procedures  to  accomplish  it? 

22.  To  what  extent  are  you  able  to  determine  how  well  you  are  doing  your  job 
without  feedback  from  anyone  else? 

23.  To  what  extent  do  additional  duties  interfere  with  the  performance  of 
your  primary  job? 

24.  To  what  extent  do  you  have  adequate  tools  and  equipment  to  accomplish 
your  job? 

25.  To  what  extent  is  the  amount  of  work  space  provided  adequate? 

26.  To  what  extent  does  your  job  provide  the  chance  to  know  for  yourself 

when  you  do  a  good  job,  and  to  be  responsible  for  your  own  work? 

27.  To  what  extent  does  doing  your  job  well  affect  a  lot  of  people? 

28.  To  what  extent  does  your  job  provide  you  with  the  chance  to  finish  com¬ 

pletely  the  piece  of  work  you  have  begun? 


5  =  To  a  fairly  large  extent 

6  =  To  a  great  extent 

7  =  To  a  very  great  extent 


1  =  Not  at  all 

2  =  To. a  very  little  extent 

3  =  To  a  little  extent 

4  =  To  a  moderate  extent 

29.  To  what  extent  does  your  job  require  you  to  use  a  number  of  complex 
skills? 

30.  To  what  extent  does  your  job  give  you  freedom  to  do  your  work  as  you  see 
fit? 

31.  To  vrtiat  extent  are  you  allowed  to  make  the  major  decisions  required  to 
perform  your  job  well? 

32.  To  what  extent  are  you  proud  of  your  job? 

33.  To  what  extent  do  you  feel  accountable  to  your  supervisor  in  accomplish¬ 
ing  your  job? 

34.  To  what  extent  do  you  know  exactly  what  is  expected  of  you  in  performing 
your  job? 

35.  To  what  extent  are  your  job  performance  goals  difficult  to  accomplish? 

36.  To  what  extent  are  your  job  performance  goals  clear? 

37.  To  what  extent  are  your  job  performance  goals  specific? 

38.  To  what  extent  are  your  job  performance  goals  realistic? 

39.  To  what  extent  do  you  perform  the  same  tasks  repeatedly  within  a  short 
period  of  time? 

40.  To  what  extent  are  you  faced  with  the  same  type  of  problem  on  a  weekly 
basis? 

* 

41.  To  what  extent  are  you  aware  of  promotion/advancement  opportunities  thct 
affect  you? 

42.  To  what  extent  do  co-workers  in  your  work  group  maintain  high  standards 
of  performance? 

43.  To  what  extent  do  you  have  the  opportunity  to  progress  up  your  career 
ladder? 

44.  To  vrtiat  extent  are  you  being  prepared  to  accept  increased  responsibil¬ 
ity? 

45.  To  what  extent  do  people  who  perform  well  receive  recognition? 

46.  To  what  extent  does  your  work  give  you  a  feeling  of  pride? 
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5  =  To  a  fairly  large  extent 

6  =  To  a  great  extent 

7  =  To  a  very  great  extent 


1  =  Not  at  al 1 

2  =  To  a  very  little  extent 

3  =  To  a  little  extent 

4  =  To  a  moderate  extent 

47.  To  what  extent  do  you  have  the  opportunity  to  learn  skills  which  will 
improve  your  promotion  potential? 

48.  To  what  extent  do  you  have  the  necessary  supplies  to  accomplish  your 
job? 

49.  To  what  extent  do  details  (tasks  not  covered  by  primary  or  additional 
duty  descriptions)  interfere  with  the  performance  of  your  primary  job? 

50.  To  what  extent  does  a  bottleneck  in  your  organization  seriously  affect 
the  flow  of  work  either  to  or  from  your  group? 


JOB  DESIRES 


The  statements  below  deal  with  job  related  characteristics.  Read  each  state¬ 
ment  and  choose  the  response  which  best  represents  how  much  you  would  like  to 
have  each  characteristic  in  your  job. 

In  my  job,  I  would  like  to  have  the  characteristics  described: 

1  *  not  at  all  5  =  A  large  amount 

2  =  A  slight  amount  6  »  A  very  large  amount 

3  =  A  moderate  amount  7  -  An  extremely  large  amount 

4  =  A  fairly  large  amount 

51.  Opportunities  to  have  independence  in  my  work. 

52.  A  job  that  is  meaningful. 

53.  An  opportunity  for  personal  growth  in  my  job. 

54.  Opportunities  in  my  work  to  use  my  skills. 

55.  Opportunities  to  perform  a  variety  of  tasks. 

56.  A  job  in  which  tasks  are  repetitive. 

57..  A  job  in  which  tasks  are  relatively  easy  to  accomplish. 
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SUPERVISION 


The  statements  below  describe  characteristics  of  managers  or  supervisors. 
Indicate  your  agreement  by  choosing  the  phrase  which  best  represents  your 
attitude  concerning  your  supervisor. 

1  =  Strongly  disagree  5  =  Slightly  agree 

2  =  Moderately  disagree  6  =  Moderately  agree 

3  =  Slightly  disagree  7  =  Strongly  agree 

4  =  Neither  agree  nor  disagree 

Select  the  coresponding  number  for  each  statement  and  enter  it  on  the  separate 
response  sheet. 

58.  My  supervisor  is  a  good  planner. 

59.  My  supervisor  sets  high  performance  standards. 

60.  My  supervisor  encourages  teamwork. 

61.  My  supervisor  represents  the  group  at  all  times. 

62.  My  supervisor  establishes  good  work  procedures. 

63.  My  supervisor  has  made  his  responsibilities  clear  to  the  group. 

64.  My  supervisor  fully  explains  procedures  to  each  group  member. 

65.  My  supervisor  performs  well  under  pressure. 

66.  My  supervisor  takes  time  to  help  me  when  needed. 

67.  My  supervisor  asks  members  for  their  ideas  on  task  improvements. 

68.  My  supervisor  explains  how  my  job  contributes  to  the  overafl  mission. 

69.  My  supervisor  helps  me  set  specific  goals. 

70.  My  supervisor  lets  me  know  when  I  am  doing  a  good  job. 

71.  My  supervisor  lets  me  know  when  I  am  doing  a  poor  job. 

72.  My  supervisor  always  helps  me  improve  n\y  performance. 

73.  My  supervisor  insures  that  I  get  job  related  training  when  needed. 

74.  My  job  performance  has  improved  due  to  feedback  received  from  my 
supervisor. 


75.  When  I  neet  technical  advice,  I  usually  go  to  rr\y  supervisor. 

76.  ^  supervisor  frequently  gives  me  feedback  on  how  well  I  am  doing  ny  job. 


WORK  GROUP  PRODUCTIVITY 

The  statements  below  deal  with  the  output  of  your  work  group.  The  term  "your 
work  group"  refers  to  you  and  your  co-workers  who  work  for  the  same  supervisor. 
Indicate  your  agreement  with  the  statement  by  selecting  the  phrase  which  best 
expresses  your  opinion. 

1  »  Strongly  disagree  4  =  Neither  agree  nor  disagree 

2  =  Moderately  disagree  5  =  Slightly  agree 

3  =  Slightly  disagree  6  =  Moderately  agree 

7  =  Strongly  agree 

Select  the  corresponding  number  for  each  statement  and  enter  it  on  the  separate 
response  sheet. 

77.  The  quantity  of  output  of  your  work  group  is  very  high. 

78.  The  qua! ity  of  output  of  your  work  group  is  very  high. 

79.  When  high  priority  work  arises,  such  as  short  suspenses,  crash  programs, 
and  schedule  changes,  the  people  in  njy  work  group  do  an  outstanding  job  in 
handling  these  situations. 

80.  Your  work  group  always  gets  maximum  output  from  available  resources  (e.g., 
personnel  and  material). 

81.  Your  work  group's  performance  in  comparison  to  similar  work  groups  is  very 
high. 

ORGANIZATION  CLIMATE 

Below  are  items  which  describe  characteristics  of  your  organization. ,  The  term 
"your  organization"  refers  to  your  squadron  or  staff  agency.  Indicate  your 
agreement  by  choosing  the  phrase  which  best  represents  your  opinion  concerning 
your  organization. 

1  =  Strongly  disagree  5  =  Slightly  agree 

2  =  Moderately  disagree  6  =  Moderately  agree 

.  3  =  Slightly  disagree  7  =  Strongly  agree 

4  =  Neither  agree  nor  disagree 

Select  the  corresponding  number  for  each  item  and  enter  it  on  the  separate 
response  sheet. 


5  =  Slightly  agree 

6  =  Moderately  agree 

7  =  Strongly  agree 


1  =  Strongly  disagree 

2  -  Moderately  disagree 

3  =  Slightly  disagree 

4  =  Neither  agree  or  disagree 

82.  Ideas  developed  by  my  work  group  are  readily  accepted  by  management 
personnel  above  my  supervisor. 

83.  My  organization  provides  all  the  necessary  information  for  me  to  do  my  job 
effectively. 

84.  My  organization  provides  adequate  information  to  my  work  group. 

85.  My  work  group  is  usually  aware  of  important  events  and  situations. 

86.  My  complaints  are  aired  satisfactorily. 

87.  My  organization  is  very  interested  in  the  attitudes  of  the  group  members 
toward  their  jobs. 

88.  My  organization  has  a  very  strong  interest  in  the  welfare  of  its  people. 

89.  I  am  very  proud  to  work  for  this  organization. 

90.  I  feel  responsible  to  my  organization  in  accomplishing  its  mission. 

91.  The  information  in  my  organization  is  widely  shared  so  that  those  needing 
it  have  it  available. 

92.  Personnel  in  my  unit  are  recognized  for  outstanding  performance. 

93.  I  am  usually  given  the  opportunity  to  show  or  demonstrate  my  work  to 
others. 

94.  There  is  a  high  spirit  of  teamwork  among  my  co-workers. 

95.  There  is  outstanding  cooperation  between  work  groups  of  my  organization. 

96.  My  organization  has  clear-cut  goals. 

97.  I  feel  motivated  to  contribute  my  best  efforts  to  the  mission  of  my 
organization. 

98.  My  organization  rewards  individuals  based  on  performance. 

99.  The  goals  of  my  organization  are  reasonable. 

100.  My  organization  provides  accurate  information  to  my  work  group. 
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.’03  RELATED  ISSUES 


The  items  below  are  used  to  determine  how  satisfied  you  are  with  specific  job 
related  issues.  Indicate  your  degree  of  satisfaction  or  dissatisfaction  with 
each  issue  by  choosing  the  most  appropriate  phrase. 

1  =  Extremely  dissatisfied  5  -  Slightly  satisfied 

2  =  Moderately  dissatisfied  6  =  Moderately  satisfied 

3  =  Slightly  dissatisfied  7  =  Extremely  satisfied 

4  =  Neither  satisfied  nor  dissatisfied 

Select  the  corresponding  number  for  each  question  and  enter  it  on  the  separate 
response  sheet. 


101.  Feeling  of  Helpfulness 

The  chance  to  help  people  and  improve  their  welfare  through  the  perforiiv 
ance  of  my  job.  The  importance  of  my  job  performance  to  the  welfare  of 
others. 

102.  Co-Morker  Relationship 

My  amount  of  effort  compared  to  the  effort  of  n\y  co-workers,  the  extent 
to  which  my  co-workers  share  the  load,  and  the  spirit  of  teamwork  which 
exists  among  my  co-workers. 

103.  Family  Attitude  Toward  Job 

The  recognition  and  the  pride  my  family  has  in  the  work  I  do. 

104.  On-the-Job  Training  (OJT) 

The  OJT  instructional  methods  and  instructors'  competence. 

105.  Technical  Training  (Other  than  OJT) 

The  technical  training  I  have  received  to  perform  my  current  job. 

106.  Work  Schedule 

My  work  schedule;  flexibility  and  regularity  of  n\y  work  schedule;  the 
number  of  hours  I  work  per  week. 

107.  Job  Security 

108.  Acquired  Valuable  Skills 

The  chance  to  acquire  valuable  skills  in  rriy  job  which  prepare  me  for 
future  opportunities. 


Appendix  B 


OAP  Standard  Response  Sheet 
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CODE  AND  WRITE  IT  IN  THE  SPACE  PROVIDED  ON  THE  COMMANDER'S  QUESTIONNAIRE. 

(PAUSE)  THIS  SURVEY  SHOULD  TAKE  ABOUT  FIVE  MINUTES.  READ  THE  INSTRUCTIONS 
AND  BEGIN.  (WAIT  FIVE  MINUTES)  IF  YOU  HAVE  NOT  COMPLETED  THE  COMMANDER'S 
QUESTIONNAIRE,  PLEASE  PUT  IT  ASIDE  WHILE  I  GIVE  INSTRUCTIONS  FOR  THE  SECOND 
SURVEY.  (GO  TO  PART  II  OF  THIS  SCRIPT) 

B.  THE  ORGANIZATIONAL  ASSESSMENT  PACKAGE  SURVEY  THAT  YOU  WILL  BE  TAKING 
CONSISTS  OF  109  STATEMENTS  DEVELOPED  FOR  COMPUTER  PROCESSING  AND  WHICH  SAMPLE 
CERTAIN  CONCERNS  THROUGHOUT  THE  AIR  FORCE.  PLEASE  DO  NOT  START  UNTIL  WE  GIVE 
FURTHER  INSTRUCTIONS.  (PASS  OUT  SURVEY  BOOKLETS  AND  RESPONSE  SHEETS.) 

SLIDE  #3:  EACH  OF  YOU  SHOULD  HAVE  AM  APPOINTMENT  SLIP  SUCH  AS  THE  ONE  SHOWN 
ON  THE  SLIDE.  AT  THIS  TIME  TAKE  OUT  YOUR  APPOINTMENT  SLIPS.  (PAUSE  AND  SLIDE 
#4). 

II.  OAP  RESPONSE  SHEET  INSTRUCTIONS: 

NEXT,  TAKE  YOUR  COMPUTER  ANSWER  SHEET,  TURN  IT  SIDEWAYS,  AND  LOCATE  SECTION  ONE 
AT  THE  TOP  RIGHT  CORNER  OF  THE  SHEET,  THIS  BACKGROUND  INFORMATION  IS  NECESSARY 
FOR  US  TO  LOOK  AT  AIR  FORCE  WIDE-ISSUES  AND  NOT  TO  IDENTIFY  INDIVIDUALS.  FOR 
EXAMPLE,  IF  WE  WANT  TO  KNOW  WHAT  AFSCs  OR  CERTAIN  OTHER  GROUPS  THINK  ABOUT  A 
PARTICULAR  SUBJECT,  THEN  WE  CAN  EXTRACT  THAT  DATA  FROM  THE  COMPUTER  ANALYSIS. 

WE  WILL  COMPLETE  SECTION  ONE  TOGETHER,  SO  PLEASE  WAIT  FOR  MY  INSTRUCTIONS. 

REFER  TO  YOUR  APPOINTMENT  SLIP.  LOCATE  THE  10-CHARACTER,  ALPHA-NUMERIC  SURVEY 
CODE.  WRITE  THESE  LETTERS  AND  NUMBERS  FORM  LEFT  TO  RIGHT  IN  THE  10  SQUARES 
LABELED  "SUPERVISORS  CODE"  AND  "WORK  GROUP  CODE"  ACROSS  THE  TOPS  OF  THE  BLOCKS. 
(REFER  TO  SLIDE) 

THERE  SHOULD  NOT  BE  ANY  ZEROES  IN  YOUR  10-CHARACTER  CODE.  ALSO,  THE  FIRST  FIVE 
CHARACTERS  OF  NON-SUPERVISORS'  CODES  SHOULD  BE  ALL  "Zs".  PLEASE  INSURE  WHEN  YOU 
PRINT  YOUR  Zs  THAT  THEY  DO  NOT  LOOK  LIKE  THE  NUMBER  "TWO".  AFTER  PRINTING  IN 
THE  10-CHARACTER  CODE,  DARKEN  THE  OVALS  IMMEDIATELY  BELOW  THE  SQUARES  THAT 
CORRESPOND  TO  YOUR  SURVEY  CODE.  (PAUSE). 

UNDER  "SEX"  PRINT  THE  LETTER  "M"  OR  "F"  IN  THE  SQUARE  AND  BALCKEN  THE  "M"  OVAL 
FOR  MALE,  OR  THE  "F"  FOR  FEMALE.  (PAUSE). 
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SLIDE  #  1 :  WELCOME 

PASS  OUT  #2  PENCILS  AND  CHECK  FOR  APPOINTMENT  SLIPS  AS  PEOPLE  ENTER. 

I.  INTRODUCTION;  GOOD  MORNING/ AFTERNOON.  I  AM _ FROM  THE 

LEADERSHIP  AND  MANAGEMENT  DEVELOPMENT  CENTER,  MAXWELL  AFB,  ALA.  WE  WORK 
DIFFERENTLY  FROM  MOST  TEAMS  THAT  COME  TO  YOUR  ORGANIZATION.  WE  VISIT  AN 
ORGANIZATION  BY  INVITATION  ONLY  AND,  IN  THIS  CASE,  WE  ARE  HERE  AT  THE  PERSONAL 

INVITATION  OF  YOUR  COMMANDER,  _ . 

SLIDE  #2.  ALSO  WE  DO  NOT  REPORT  THE  RESULTS  OF  OUR  WORK  BEYOND  YOUR  COMMANDER; 
WHAT  WE  IDENTIFY  HERE  WE  LEAVE  HERE.  YOU  ARE  TAKING  A  SURVEY  THIS  MORNING/ 
AFTERNOON  SO  THAT  YOUR  OPINIONS  MAY  BE  SHARED  WITH  THE  ORGANIZATION  LEADERS; 
HOWEVER,  YOU  REMAIN  ANONYMOUS.  NO  ONE  BUT  PEOPLE  FROM  THE  LEADERSHIP  AND 
MANAGEMENT  DEVELOPMENT  CENTER  WILL  SEE  YOUR  RESPONSE  SHEETS.  '  WE  WILL  RETURN 

In _ AT  WHICH  TIME  SURVEY  RESULTS  WILL  BE  FED  BACK  TO  YOU.  SINCE 

YOU  WILL  BE  MAKING  A  VALUABLE  CONTRIBUTION,  IT  IS  EXTREMELY  IMPORTANT  THAT  YOU 
BE  VERY  CAREFUL  IN  COMPLETING  YOUR  RESPONSE  SHEET.  BE  SURE  YOU  MAKE  ONLY  ONE 
RESPONSE  TO  EACH  STATEMENT  AND  DO  NOT  LEAVE  ANY  STRAY  PENCIL  MARKS  ON  THE  RE¬ 
SPONSE  SHEET. 

(SELECT  EITHER  "A"  OR  "B") 

A.  YOU  WILL  COMPLETE  TWO  SURVEYS.  THE  FIRST  SURVEY  IS  A  COMMANDERS'S 

QUESTIONAIRE;  IT  CONTAINS  _ QUESTIONS  THAT  ALLOW  YOU  TO  EXPRESS  YOUR  FEELINGS 

ON  PARTICULAR  ISSUES  OF  INTEREST  TO  YOUR  COMMANDER.  THE  SECOND  SURVEY  IS  THE 
ORGANIZATIONAL  ASSESSMENT  PACKAGE  SURVEY;  IT  CONTAINS  109  STATEMENTS  DEVELOPED 
FOR  COMPUTER  PROCESSING  AND  WHICH  SAMPLE  CERTAIN  CONCERNS  THROUGHOUT  THE  AIP 
FORCE.  PLEASE  DO  NOT  START  EITHER  SURVEY  UNTIL  WE  GIVE  INSTRUCTIONS.  (PASS 
OUT  COMMANDER'S  QUESTIONAIRE,  OAP  SURVEY  BOOKLET,  AND  RESPONSE  SHEETS) 

SLIDE  #3.  WE  WILL  BEGIN  WITH  THE  COMMANDER'S  QUESTIONNAIRE.  REFER  TO  YOUR 
APPOINTMENT  SLIP  AS  SHOWN  ON  THE  SCREEN.  FIND  YOUR  10-CHARACTER  SURVEY 
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IN  THE  "YOUR  ARE"  BLOCK,  WRITE  YOUR  AGE  IN  THE  SQUARES  AND  BLACKEN  THE  CORRESPONDING 

i 

OVALS.  (PAUSE). 

IN  THE  BLOCK  TITLED  "YOU  ARE,"  DARKEN  THE  OVAL  TO  THE  RIGHT  OF  THE  WORD  THAT 
MOST  CORRECTLY  IDENTIFIES  YOUR  STATUS.  FOR  EXAMPLE,  OFFICER,  ENLISTED,  WG  OR  WS. 
(PAUSE) 

COMPLETE  THE  "YOUR  PAY  GRADE"  BLOCK  BY  WRITING  YOUR  PAY  GRADE  IN  THE  SQUARES  AND 
BLACKEN  THE  OVALS  THAT  CORRESPOND  TO  YOUR  PAY  GRADE.  FOR  EXAMPLE,  "ZERO  FOUR" 

FOR  PAY  GRADE  GS-4,  WG-4,  0-4,  OR  E-4.  (PAUSE) 

SLIDE  #5:  AFSC  BLOCKS 

PLEASE  REVIEW  THE  SLIDE  BEFORE  WE  COMPLETE  THE  AFSC  BLOCKS  AND  NOTE  THE  INSTRUCTION.:. 
(PAUSE)  IN  THE  "PRIMARY"  AND  "DUTY"  BLOCKS,  WRITE  YOUR  AFSCs  IN  THE  SQUARES  AND 
DARKEN  THE  CORRESPONDING  OVALS  TO  INCLUDE  ANY  PREFIX  AND/OR  SUFFIX.  LEAVE  THE 
FIRST  BLOCK  BLANK  IF  IT  DOES  NOT  CONTAIN  A  SUFFIX.  IF  YOU  HAVE  A  FOUR-DIGIT  AFSC. 
ENTER  A  "ZERO"  IN  THE  FIRST  NUMERIC  POSITION  AND  DARKEN  THE  CORRESPONDING  OVAL. 

THIS  COMPLETES  THE  BACKGROUND  INFORMATION. 

HI.  OAP  CONTENTS:  OPEN  THE  COVER  PAGE  OF  THE  SURVEY  BOOKLET  AND  NOTE  THE 
STANDARD  PRIVACY  ACT  STATEMENT.  (PAUSE)  THE  NEXT  TWO  PAGES  CONTAIN  GENERAL 
INFORMATION  AND  INSTRUCTIONS.  NOTE  THE  KEY  DEFINITIONS  FOR  SUPERVISOR,  WORK 
GROUP,  AND  ORGANIZATION.  „ 

SLIDE  #6:  DEFINITIONS.  (READ  EACH  DEFINITION  AND  POINT  OUT  THAT  THE  DEFINITIONS 
SLIDE  WILL  REMAIN  ON  THE  SCREEN  WHILE  THE  PEOPLE  ARE  TAKING  THE  SURVEY) 

SUPERVISOR— THE  PERSON  TO  WHOM  YOU  REPORT  DIRECTLY. 

WORK  GROUP— ALL  PERSONS  WHO  REPORT  TO  THE  SAME  SUPERVISOR  AS  YOU. 

ORGANIZATION— YOUR  SQUADRON.  HOWEVER,  FOR  THOSE  INDIVIDUALS  WHO  WORK  STAFF  OR 
SUPPORT  AGENCIES,  THE  DIVISION  OR  DIRECTORATE  WOULD  BE  THE  OR¬ 
GANIZATION. 
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SLIDE  #7;  RESPONSE  SHEET  EXAMPLE 

REVIEW  THE  SAMPLE  STATEMENT  ON  PAGE  111.  (PAUSE  WHILE  THEY  REVIEW)  IF  YOU 
MODERATELY  AGREE  WITH  THE  SAMPLE  STATEMENT.  YOU  WOULD  HAVE  DARKENED  OVAL  "6" 

ON  THE  ANSWER  SHEET.  IF  THE  STATEMENT  DID  NOT  APPLY  TO  YOUR  WORK  GROUP,  YOU 
WOULD  HAVE  DARKENED  THE  "ZERO"  OVAL  AS  SHOWN  ON  THE  SCREEN.  ARE  THERE  ANY 
QUESTIONS? 

BE  SURE  TO  GIVE  ONLY  ONE  RESPONSE  TO  EACH  STATEMENT  AND  DO  NOT  LEAVE  STRAY 
MARKS  ON  THE  RESPONSE  SHEET.  (IF  AN  ADDITIONAL  COMMENT  SHEET  HAS  BEEN  PROVIDED, 
POINT  THIS  OUT  TO  THEM.) 

REMEMBER,  THIS  IS  NOT  A  TEST— THERE  ARE  NO  RIGHT  OR  WRONG  ANSWERS.  YOUR 
RESPONSES  WILL  MERELY  EXPRESS  YOUR  OPINIONS.  IF  YOU  HAVE  ANY  QUESTIONS  DURING 
THE  SURVEY,  PLEASE  RAISE  YOUR  HAND  AND  WE  WILL  HELP  YOU.  WHEN  YOU  COMPLETE  THE 
SURVEY,  YOU  MAY  LEAVE.  AS  YOU  DEPART,  PLEASE  DO  SO  QUIETLY  AND  PLACE  YOUR 
SURVEY  MATERIAL  AND  APPOINTMENT  SLIPS  IN  THE  APPROPRIATE  BOXES  NEAR  THE  EXITS. 
THANK  YOU  FOR  YOUR  PARTICIPATION.  IF  THERE  ARE  NO  QUESTIONS  AT  THIS  TIME,  PLEASE 
BEGIN. 

(PLACE  SLIDE  #6  ON  THE  SCREEN  AND  LEAVE  THERE  DURING  THE  SURVEY) 
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ORGANIZATIONAL  ASSESSMENT  PACKAGE  OUTPUT  I  III  FACTOR:  DEMOGRAPHIC  (MOT  A  STATISTICAL  FACTOR) 
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VOTE-  Variable  008,  statement  11  was  added  to  the  OAP  on  19  Jan  80  and  replaced  .  1.  yes  2.  No  3.  Not  sure 

rarlable  014  which  appears  on  page  3.  Although  no  longer  used  Variable 
014  Is  still  shown  because  data  collected  from  about  25,000  samples  for 
this  variable  Is  still  in  the  data  base.  it 


VARIABLE  STATEMENT 

NUMBER  NUMBER  STATEMENT 


FACTOR  802:  TASK  SIGNIFICANCE 


FACTOR  812:  TASK  CHARACTERISTIC 


To  uhet  t»tent  ere  you  ellowd  to  Bike  the 
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FACTOR  822:  JOB  RELATED  SATISFACTION   Mil  FACTUM*:  GENERAL  QRGAHIZAT  ICN'Al  CLIMATE 
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